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Reply to Office Action of February 23, 2007 

This listing of claims will replace all prior versions, and listings, of claims in the 
application: 

Listing of Claims: 

Claims 1-45 (Canceled). 

Claim 46 (Currently Amended): A computer-implemented method for editing a 
video stream to combine augmented virtual images with video data utilizing 
augmont o d roal i ty (AR) t o chnology to combino roal and virtua l wor l dc tog o thor to 
provido an intorfaco for a uoor to conco -a nd interact with virtua l objocts in tho r o ai 
wefld, the method comprising: 

providing video data comprising images of a moving model plane having 
markers, said markers for calibrating a camera to track the motion of the model 
plane; 

determining a pose of the moving model plane according to the markers in 
the video data and calibration results; 

creating a three dimensional image data model of a product in a pose 
corresponding to the pose of the moving model plane, wherein the orientation of 
the markers determines the orientation of the three dimensional image data 
model; 

determining an image correspondence between the camera calibration 
results associated with the moving model plane and the three-dimensional 
image data model; 

rendering a 3D model of the product based on the image correspondence; 

generating an augmented reality video by superimposing the rendered 3D 
model of the product on the moving model piane in the video data ; and 
displaying the augmented reality video . 

Claim 47 (Canceled). 
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Claim 48 {Previously Presented): The computer-implemented method of claim 
46, further comprising scaling the three-dimensional image data model to the 
model plane according to the markers. 

Claim 49 (Canceled). 

Claim 50 (Previously Presented) The computer-implemented method of claim 
46, further comprising encoding hyperlink information into the augmented reality 
video, wherein the hyperlink information points to data corresponding to the 
product. 

Claim 51 (Previously Presented) The computer-implemented method of claim 
50, wherein encoded hyperlink information is activated by selecting a rendering 
of the three-dimensional image data model from the augmented reality video. 

Claim 52 (Previously Presented) The computer-implemented method of claim 
46, wherein the video data includes images of a human manipulating the pose of 
the model plane, wherein a rendering of the three-dimensional image data model 
is manipulated according to the pose of the model plane. 

Claim 53 (Previously Presented) The computer-implemented method of claim 46 
wherein the augmented reality video is produced in real time. 

Claim 54 (Currently Amended) A program storage device readable by machine, 
tangibly embodying a program of instructions executable by the machine to 
perform method steps for editing a video stream to combine augmented virtual 
images with video data utiliz i ng augmontod reality (AR) tochnology to combin o 
rea l and virtual wor l do togothor to prov i de an interface for a ucor to oons o and 
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interact with virtual objects i n tho real world , the method , the method steps 
comprising: 

providing video data comprising images of a moving mode! plane having 
markers, said markers for calibrating a camera to track the motion of the model 
plane; 

determining a pose of the moving model plane according to the markers in 
the video data and calibration results; 

creating a three dimensional image data model of a product in a pose 
corresponding to the pose of the moving model plane, wherein the orientation of 
the markers determines the orientation of the three dimensional image data 
model;; 

determining an image correspondence between the camera calibration 
results associated with the moving model plane and the three-dimensional image 
data model; 

rendering a 3D model of the product based on the image correspondence; 

generating an augmented reality video by superimposing the rendered 3D 
model of the product on the moving model plane in the video data ; and 
displaying the augmented reality video . 

Claim 55 (Canceled). 

Claim 56 (Previously Presented): The machine to perform method of claim 54, 
further comprising scaling the three-dimensional image data model to the model 
plane according to the markers. 

Claim 57 (Canceled). 
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Claim 58 (Previously Presented): The machine to perform method of claim 54, 
further comprising encoding hyperiink information into the augmented reality 
video, wherein the hyperlink information points to data corresponding to the 
product. 

Claim 59 (Previously Presented): The machine to perform method of claim 58, 
wherein encoded hyperlink information is activated by selecting a rendering of 
the three-dimensional image data model from the augmented reality video. 

Claim 60 (Previously Presented): The machine to perform method of claim 54, 
wherein the video data includes images of a human manipulating the pose of the 
model plane, wherein a rendering of the three-dimensional image data model is 
manipulated according to the pose of the model plane. 

Claim 61 (Currently Amended): A computer-implemented method for producing 
an augmented reality video comprising: 

providing video data comprising images of a moving model plane having a 
marker, wherein the model plane is a substantialiy rectangular plane and the 
marker is a graphic disposed on an upper surface thereof; 
providing a throo dimens i ona l imago data modol of a product; 

determining a pose of the moving model plane according to the markers in 
the video data, the pose highlighting a particular perspective of the moving model 
plane ; 

creating a three dimensional image data model of a product in a p ose 
correspondin g to the pose of the moving model plane, wherein orientation of the 
markers determines the orientation of the three dimensional image data model: 

determining an image correspondence between the moving model plane 
having markers and the three-dimensional image data model; 
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producing the augmented reality video by rendering the three-dimensional 
image data mode! of the product superimposed on the moving model plane in the 
video data, wherein a rendering of the three-dimensional image data model has 
substantially the pose of the moving model plane ; and 

displaying the augmented reality video . 

Claim 62 (Currently Amended): A computer-implemented method for producing 
an augmented reality video comprising: 

providing video data comprising images of a moving model plane having a 
marker, wherein the model plane is a substantially rectangular plane and the 
marker is a graphic disposed on an upper surface thereof; 

provid i ng a throo dim o noiona l imago data model of a product, whoroin tho 
throo dimonsiona i imag o data modol i s an an i mation; 

determining a pose of the moving model plane according to the markers in 
the video data, the pose highlight ing a particular perspective of the moving model 
plane ; 



creating a thre e dimensional image data model of a product in p oses 
corresponding to the po ses of the moving model plane, wherein the three 
dimensional image da ta model is an animation and the orientation of the markers 
determines the orient ation of the three dimensional image data model: 

determining an image correspondence between the moving model plane 
having markers and the three-dimensional image data model; 

producing the augmented reality video by rendering the three-dimensional 
image data model of the product superimposed on the moving model plane in the 
video data, wherein a rendering of the three dimensional image data mode! has 
substantially the pose of the moving model plane; 

encoding hyperlink information into the augmented reality video, wherein 
the hyperlink information points to data corresponding to the product, wherein 
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encoded hyperlink information is activated by selecting the rendering of the 

three-dimensional image data model from the augmented reality video; 

converting the augmented reality video into a streaming video format; and 
streaming the augmented reality video having the streaming video format 

over a communications network to a computer for displaying the augmented 

reality video ; and 

displaying the augmented reality video . 
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